Skip to content

Add cupy backend for GPU support #1048

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 40 commits into
base: master
Choose a base branch
from
Open

Add cupy backend for GPU support #1048

wants to merge 40 commits into from

Conversation

@s3alfisc
Copy link
Member

@s3alfisc s3alfisc commented Oct 23, 2025

We add a cupy backend https://docs.cupy.dev/en/latest/index.html to run FWL on the GPU.

Strategy: "direct" FWL OLS residualization instead of iterative solver.

Should help with problems with slow convergence.

Only tested on CPU via scipy, where results are matching. Cannot test on GPU as no Mac / no access to Cuda GPU.

Note: we can build a backend using a similar strategy via JAX and torch.

import pyfixest as pf

data = pf.get_data()

fit_rust = pf.feols("Y ~ X1 + X2  | f1", data = data, demeaner_backend="rust")
fit_cupy = pf.feols("Y ~ X1 + X2  | f1", data = data, demeaner_backend="cupy")

(pf.etable([fit_rust, fit_cupy], type = "md", digits = 8))
index                    est1             est2
------------  ---------------  ---------------
depvar                      Y                Y
----------------------------------------------
X1            -0.94952557***   -0.94952557***
                 (0.06637281)     (0.06637281)
X2            -0.17422527***   -0.17422527***
                 (0.01759569)     (0.01759569)
f1                          x                x
----------------------------------------------
Observations              997              997
----------------------------------------------
S.E. type                 iid              iid
R2                 0.48899391       0.48899391
R2 Within          0.23875790       0.23875790
----------------------------------------------

@s3alfisc s3alfisc linked an issue Oct 23, 2025 that may be closed by this pull request
pyfixest is MUCH slower than (Stata + Julia) reghdfejl #1042
Open
@s3alfisc s3alfisc mentioned this pull request Oct 23, 2025
Open
@codecov
Copy link

codecov bot commented Oct 23, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 23.94366% with 108 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pyfixest/estimation/cupy/demean_cupy_.py 21.13% 97 Missing ⚠️
pyfixest/estimation/demean_.py 0.00% 6 Missing ⚠️
pyfixest/estimation/backends.py 54.54% 5 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (70a154c) and HEAD (6388b11). Click for more details.

HEAD has 2 uploads less than BASE
Flag BASE (70a154c) HEAD (6388b11)
core-tests 2 0
Flag Coverage Δ
core-tests ?
tests-vs-r 15.82% <23.94%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
pyfixest/estimation/estimation.py 9.19% <ø> (-81.61%) ⬇️
pyfixest/estimation/literals.py 68.75% <100.00%> (-18.75%) ⬇️
pyfixest/estimation/vcov_utils.py 19.01% <100.00%> (-20.71%) ⬇️
pyfixest/estimation/backends.py 72.41% <54.54%> (-10.92%) ⬇️
pyfixest/estimation/demean_.py 12.09% <0.00%> (-41.30%) ⬇️
pyfixest/estimation/cupy/demean_cupy_.py 21.13% <21.13%> (ø)

... and 38 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@s3alfisc
Copy link
Member Author

s3alfisc commented Oct 23, 2025 •
edited
Loading

TODO

  • add tests
  • create GPU env
  • maybe fix numpy to pandas to sparse array conversion? could lead to poor performance

@review-notebook-app
Copy link

review-notebook-app bot commented Oct 31, 2025

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@s3alfisc
Copy link
Member Author

s3alfisc commented Nov 2, 2025

pre-commit.ci autofix

pre-commit-ci bot and others added 15 commits November 2, 2025 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

pyfixest is MUCH slower than (Stata + Julia) reghdfejl

2 participants

@s3alfisc